Skeleton-Based Action and Gesture Recognition for Human-Robot Collaboration

Human action recognition plays a major role in enabling an effective and safe collaboration between humans robots. Considering for example collaborative assembly task, the human worker can use gestures to communicate with robot while exploit recognized actions anticipate next steps process, improving safety overall productivity. In this work, we propose novel framework based on 3D pose estimation ensemble techniques. such framework, first estimate coordinates of hands body joints by means OpenPose RGB-D data. The estimated are then fed set graph convolutional networks derived from Shift-GCN, one network each (i.e., body, left hand right hand). Finally, using approach average output scores all predict final action. proposed was evaluated dedicated dataset, named IAS-Lab Collaborative HAR which includes both commonly used human-robot tasks. experimental results demonstrated how different models helps accuracy robustness system.